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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)Q This action is FINAL. 2b)S This action is non-final. ■ 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-14 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) £3 Claim(s) 1-14 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10)[X] The drawing(s) filed on 24 February 2004 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)£3 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1. D Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3.S Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Information Disclosure Statement 

1 . The information disclosure statement filed 1 5 July 2004 fails to comply with 37 
CFR 1.98(a)(3) because it does not include a concise explanation of the relevance, as it 
is presently understood by the individual designated in 37 CFR 1.56(c) most 
knowledgeable about the content of the information, of each patent listed that is not in 
the English language. It has been placed in the application file, but the information 
referred to therein has not been considered. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claims 1 , 3, 4, 7, 8, 1 0, 1 1 , 14 are rejected under 35 U.S.C. 1 02(b) as being 
anticipated by Hanna et al. (US Patent No. 5,566,251). 

Hanna et al. teaches a system and method inserting a given pattern at a target 
region having a particular location with respect to a scene being viewed by an image 
sensor. 



Application/Control Number: 10/784,836 Page 3 

Art Unit: 2672 

In regards to claims 1, 7, 8, 14- 

Referring to Fig. 2, camera 200A records in real time a sequence of successive 2- 
dimensional video image frames of physical three-dimensional objects, such as 
billboard (202), situated in background scene (204A) (claims 1, 8: a camera configured 
to acquire image information of a section of an environment) [column 4, lines 18 - 22]. 
Billboard (202) is assumed immovable, but camera (200A) is moveable both to the left 
and right and toward and away from the objects as indicated by arrowed lines (206A). 
This will result in the orientation and scale of each of the sequence of video images of 
billboard (202) in background scene (204A) changing in correspondence with the 
movement of camera (200) from one position to another, as indicated by arrow lines 
(208A) (claims 1, 8: at least one of a zoom device configured to altera size of the 
section in accordance with a zoom factor and a device configured for three-dimensional 
orientation of the a camera in accordance with a space vector) [column 4, lines 24 - 33]. 
Means 21 OA performs the functions of (1) detecting logo pattern "A" and (2) estimating 
the pose of detected logo pattern "A" (i.e., estimating the orientation, scale and 
perspective parameters of logo pattern "A" in each of the sequence of successive 2D 
video image frames of background scene (204A) with respect to the image of one or 
more reference objects, such as billboard (202) itself) in that video image frame (claims 
1,8: wherein the computer unit is configured to compute position coordinates of the 
image information based on at least one of space coordinates of the camera, the zoom 
factor, and the space vector, computer positions of representations of the image 
information on a display area of a display device; claims 7, 14: processing image 
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information) [column 4, lines 43 - 49]. Additionally, means 214, responsive to a "pose 
information" (i.e., orientation, scale, perspective distortion, etc. parameters of detected 
logo patter "A") input (215) from means 21 OA and on-time selection via input (217) of 
manually-selected size, shape, orientation, etc. parameters of logo pattern "B" itself 
performs the function of geometrically transforming the orientation and scale of logo "B" 
of foreground scene (204B) to match the orientation and scale of the estimated pose of 
logo pattern "A" in the current image frame (as indicated by (216)) (claims 1, 8: wherein 
the computer unit is configured to assign the user information to the position 
coordinates; claims 7, 14: processing use information) [column 4, lines 64 - 68; column 
5, lines 1 - 6]. The operation of video switch (212) merges the video output of camera 
(200) with the geometrically-transformed logo pattern "B" to result in output scene (218), 
wherein logo pattern "A" is replaced by inserted logo pattern "B." (claims 1, 8: an image 
processing unit for processing the image information and the user information so as to 
reproduce the image information and the user information with the display device and 
so as to insert the user information in a location on the display area at the positions of 
the representations of the image information having the position coordinates to which 
the respective user information is assigned; claims 7, 14: reproducing the image 
information and the user information with the display device, inserting the user 
information in the location on the display area in accordance with an imaging procedure) 
[column 5, lines 6 - 9]. 
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In regards to claims 3, 10 - 

Referring to Fig. 2, camera (200A) is moveable both to the left and right and toward 
and away from the objects as indicated by arrowed lines (206A). This will result in the 
orientation and scale of each of the sequence of video images of billboard (202) in 
background scene (204A) changing in correspondence with the movement of camera 
(200) from one position to another, as indicated by arrow lines (208A) [column 4, lines 
24 - 33]. Thus the movement of the camera toward the billboard (202) in the 
background scene (204A) and the scaling correspond to a zoom function. Means 21 OA 
performs the functions of (1) detecting logo pattern "A" and (2) estimating the pose of 
detected logo pattern "A" (i.e., estimating the orientation, scale and perspective 
parameters of logo pattern "A" in each of the sequence of successive 2D video image 
frames of background scene (204A) with respect to the image of one or more reference 
objects, such as billboard (202) itself) in that video image frame [column 4, lines 43 - 
49]. Additionally, means 214, responsive to a "pose information" (i.e., orientation, scale, 
perspective distortion, etc. parameters of detected logo patter "A") input (215) from 
means 21 OA and on-time selection via input (217) of manually-selected size, shape, 
orientation, etc. parameters of logo pattern "B" itself performs the function of 
geometrically transforming the orientation and scale of logo "B" of foreground scene 
(204B) to match the orientation and scale of the estimated pose of logo pattern "A" in 
the current image frame (as indicated by (216)) (claims 3, 10: configured to select and 
insert the user information as a function of the zoom factor) [column 4, lines 64 - 68; 
column 5, lines 1-6]. 
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In regards to claim 4, 11 - 

Referring to Fig. 2, the operation of video switch (212) mergers the video output of 
camera (200) with the geometrically transformed logo pattern "B" (i.e. user information 
as claimed) to result in output scene (218), wherein logo pattern "A" is replaced by 
inserted logo pattern "B". Thus, the pose of logo pattern "B" in output scene (218) 
changes in correspondence with the movement of camera (200) from one position to 
another, as indicated by arrowed lines (220) (claims 4, 11: user information comprises 
at least one of static and dynamic information) [column 5, lines 5-12]. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 2, 5, 6, 9,12, 13 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Hanna et al. (US Patent No. 5,566,251) in view of Silverbrook (US 

Patent No. 5,479,205). 

Hanna et al. teaches the limitation of claims 2, 5, 6, 9,12, 13 with the exception of 

disclosing a triggering unit and a display screen. However, Silverbrook discloses a 

video camera and recorder for combining a real-time image with a secondary image. 
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Hanna et al. discloses a system and method inserting a given pattern at a target 
region having a particular location with respect to a scene being viewed by an image 
sensor. Referring to Fig. 2, camera 200A records in real time a sequence of successive 
2-dimensional video image frames of physical three-dimensional objects, such as 
billboard (202), situated in background scene (204A) [column 4, lines 18 - 22]. 

Silverbrook discloses a camcorder system. Referring to Fig. 1 , the video 
camcorder system's circuitry (40) includes an alpha-numeric liquid crystal display (LCD) 
(61) (claims 5, 12: the camera comprises a video camera and the display device 
comprises a display screen) and various miscellaneous controls (62) such as an auto 
zoom controller (66), and a manual zoom control (70) as seen in Fig. 2 (claims 2, 9: 
comprises a triggering unit configured to trigger at least one of the camera, the zoom 
device, and the device for 3D orientation of the camera is accordance with at least one 
of the zoom factor and the space vector) [column 2, lines 23 - 30]. The circuitry (40) of 
the system receives user input via a keyboard (17) that includes an alphanumeric 
keypad (33) a touch panel (31), and animation control keys (38) as shown in Fig. 2. 
The keyboard (17) connects to a controller (16) that interprets commands and provides 
control commands onto a bus (25) that interconnects other components within the 
system (40) (claims 6, 13: the triggering unit comprises an operator interface) [column 
2, lines 43-45]. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to substitute the camera of Hanna et al. with the video 
camcorder system of Silverbrook where the output from the CCD of Silverbrook would 
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be connected to means 21 OA of Hanna et al. so that the orientation and scaling of the 
camera within the combined invention is able to be user controlled. 



Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Hanna (US Patent No. 5,488,675). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
7661. The examiner can normally be reached on Monday - Friday, 7:00am - 4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on (571) 272-7664. The fax phone number 
for the organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). J/w/ffly 

08.03.2005 mkl Ja. RICHARD fUERPE 
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